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PREFACE 


This  research  was  accomplished  under  a  memorandum  of  understanding  between 
the  Air  Force  Logistics  Management  Center  Directorate  of  Supply  and  the  USAF 
Academy  Department  of  Mathematical  Sciences,  dated  11  December  1980.  The  AFLMC 
project  number  for  this  work  was  021029. 
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SUPPLY  PERFORMANCE  MEASURES 


1.  Int roduc tion .  The  Department  of  Mathematical  Sciences  (UFMS)  investigated 
the  relationship  between  supplv  petrormance  indicators  and  measures  of  opera¬ 
tional  capability  for  ai  1-4  wing.  Data  was  provided  for  this  effort  from 
Bergstrom  AFB  and  Moodv  AFB  and  covered  the  period  October  1978  through 
December  1980. 

The  objectives  of  this  research  as  stated  in  a  memorandum  of  understand¬ 
ing  between  AFLMC  and  DIMS  were  as  follows: 

a.  Identification  of  measurements  of  supply  performance  that  are  sta¬ 
tistically  correlated  with  measures  of  operational  capability. 

b.  Determination  of  appropriate  supply  performance  goals  to  achieve  a 

stated  level  of  operational  capability. 

The  data  which  was  provided  included  monthly  measures  of  operational 
capability  as  well  as  monthly  indicators  of  supply  performance.  The  variables 
examined  in  this  study  were  as  follows; 

Operational  Capability  Level  Explanation 

NMCS  non-mission  capable  rate  due  to  only 

supply 

NMCB  non-mission  capable  rate  due  to  both 

supply  and  maintenance 

PMCS  partial-mission  capable  rate  due  to 

only  supply 

PMCB  partial-mission  capable  rate  due  to 

both  supply  and  maintenance 

Our  understanding  was  that  these  variables  were  mutually  exclusive  and  thus 
could  be  added  to  form  new  indicators.  The  only  reason  for  considering 
such  additional  indicators  was  that  they  might  provide  "measures"  which  can 
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be  better  predicted  and  controlled  than  the  original  variables.  These  pseudo¬ 
indicators  which  we  considered  were: 

SUMS  =  NMCS  +  PMC'S 
SUMB  *  NMCB  +  PMCB 
NMCX  =  NMCS  +  NMCM 
PMCX  •*  PMCS  +  PMCP 
SUMX  *  NMCS  +  NMCB  +  PMCS  +  PMCB 
They  may  be  interpreted  as  follows: 

Pseudo  Operational 

Capability  Level  Explanation 

SUMS  aircraft  capability  degraded 

supply 

SUMB  aircraft  capability  degraded 

supply  and  maintenance 

NMCX  non-mission  capable  rate  due 

PMCX  partial-mission  capable  rate 

SUMX  aircraft  capability  degraded 

Note  the  difference  in  capability  levels  that  read  "due 

those  "due  to  supply."  These  variables  were  actually  expressed  in  units  of 
percentages,  and  were  based  on  the  percentage  of  possessed  hours  that  air¬ 
craft  spent  in  the  respective  degraded  category  during  a  given  month.  The 
only  pseudo-indicator  that  proved  of  use  to  us  was  SUMX  and  this  is  the  only 
one  which  will  be  discussed  in  any  great  detail  in  this  report. 

The  supply  indicator  variables  were  as  follows: 

Supply  Indicator  Descript  ion  Units 

EOQC.A  Gross  Availability  of  EOQ  items  Percentage 

EOQNA  Net  Availability  of  EOQ  items  Percentage 


rate  due  to  only 

rate  due  to  both 

to  supply 
due  to  supply 
due  to  supply 
to  supply  only"  and 


EOQA 


EOQ  due-out s /reason  A 


Number  of  monthly 


EOQB 

EOQ  due-outs /reason  B 

Number  of  monthly 

due-outs 

EOQC 

EOQ  due-outs /reason  C 

Number  of  monthly 

due-outs 

EOQD 

EOQ  due-outs /reason  D 

Number  of  monthly 

due-outs 

EOQTII 

Total  Inventory  Investment  for 

Dollar  amount 

EOQ 

Similar  indicator  variables  were  available  for  reparable  items.  These  variables 
are  similar  to  the  EOQ  variables  except  that  their  names  are  preceeded  by  REP 
instead  of  EOQ. 

2.  Search  for  the  Best  Predictive  Models.  We  first  decided  to  use  multivariate 
regression  analysis  to  determine  the  best  subset  of  descriptive  variables  (i.e.  , 
supply  indicators)  to  be  used  to  predict  operational  capabilities.  Although  the 
overall  objective  was  to  build  a  descriptive  (i.e.,  control)  model  rather  than  a 
predictive  model,  this  was  run  in  order  to  verify  some  of  the  results  of  the 
Auburn  report^-;  and  at  the  same  time  to  examine  the  significant  correlations  and 
relationships  among  the  descriptive  variables.  As  mentioned  previously,  there 
were  fourteen  descriptive  or  explanatory  variables  which  we  utiliEed  to  ouild  a 
model.  The  sum  of  the  "REP"  and  "EOQ"  variables  was  also  suppled  as  "TOT"; 
however,  using  this  would  have  provided  no  additional  information  and  would 
have  introduced  linear  relationships  among  the  set  of  predictor  variables. 

The  Auburn  report  used  the  EOQ,  REP,  and  TOT  variables  as  separate  predictor 
sets.  We  saw  no  reason  to  do  this  and  in  fact  combined  the  EOQ  and  REP  vari¬ 
ables  into  one  set  since  they  measure  different  things. 

Of  the  four  raw  dependent  variables  investigated,  the  best  results  were 
obtained  with  NMCS  (the  non-mission  capable  rate  due  to  supply).  The  other 


variables  were  so  inconsistent  (in  terms  of  our  ability  to  build  a  model  to 
predict  them)  that  the  results  were  discarded.  The  pseudo-indicator  with 
consistent  results  was  SUMX  which  is  the  sum  of  all  aircraft  degraded  (rate) 
due  to  supply  causes.  The  remainder  of  this  report  will  deal  with  models  for 
these  two  indicators. 

A  correlation  analysis  between  the  independent  variables  and  the  two 
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dependent  variables  was  performed  with  all  27  observations  using  the  SPSS 
package.  The  Pearson  correlation  coefficients  that  were  statistically  signifi¬ 
cant  at  the  107,  level  (i.e.,  a  *  .10)  are  reported  in  Table  1.  This  table 
can  be  used  to  prepare  a  realistic  subset  of  the  descriptive  variables  (supply 
measures)  which  can  be  used  to  predict  NMCS  or  SUMX  in  a  simple  linear  multi¬ 
variate  regression  model .  Note  that  statistically  significant  correlation 
does  not  imply  a  cause  and  effect  relationship.  In  fact  several  of  the  correla¬ 
tions  intuitively  appear  to  have  the  wrong  sign  (these  are  marked  by  an  asterisk 
in  Table  1) .  For  example  the  correlation  between  NMCS  and  EOQTII  for  Moody 
AFB  is  .4366.  The  positive  coefficient  implies  that  as  we  increase  the  amount 
invested  in  inventory  we  will  Increase  the  NMCS  rate,  which  is  clearly  counter¬ 
intuitive.  This  will  be  discussed  in  more  detail  in  a  later  section. 

The  linear  correlations  among  the  independent  variables  were  also  examined. 
As  expected  the  NA  and  GA  variables  within  each  group  (EOQ  and  REP)  are  highly 
correlated  as  are  the  due  out  variables  (A,  B,  C,  and  D) .  EOQTII  and  REPTII 
were  also  highly  correlated.  Some  unexpected  results  were  that  EOQA  and  EOQB 
showed  highly  significant  correlations  with  many  of  the  REP  variables.  These 
results  were  generally  the  same  for  both  Moody  and  Bergstrom  AFB,  although  the 
interrelationships  for  the  Moody  data  were  stronger.  These  correlations  were 
also  examined  by  each  fiscal  year.  The  results  are  included  in  a  separate 
addendum  to  this  report  -  "Computer  Listings." 
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TABLE  1.  Pearson  Correlation  Coefficient  Statistically  Significant  at  ot  *  .10. 
Bergstrom  AFB  and  Moody  AFB 

CORRELATION  COEFFICIENTS 
27  Observations  -  10/78  -  12/80 

Bergstrom  AFB  Moody  AFB 


NMCS 

SUMX 

NMCS 

SUMX 

EOQGA 

EOQNA 

0.2995* 

0.3187* 

-C.2238 

-0 . 2404 

EOQA 

0.3595 

0.4645 

EOQB 

0.3142 

EOQC 

EOQD 

-0.4175* 

0.2705 

0.2626 

EOQTII 

0.4366* 

0.5104* 

REPGA 

-0.2455 

-C .3487 

-0.2647 

-0.4062 

REPNA 

-0.3070 

-0.2583 

-0.3905 

REPA 

0.4619 

0.5272 

REPB 

0.2525 

0.3675 

REPC 

REPD 

REPTII  0.2495* 


*  Apparent  incorrect  sign  for  causal  model 
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These  results  were  examined  In  order  to  avoid  using  highly  correlated 
independent  variables  together  in  the  same  model,  thereby  keeping  the  linear 


correlations  among  the  predictor  set  as  small  as  possible. 

Since  the  data  was  provided  in  the  form  of  a  time  series,  some  simple 
time  series  models,  namely,  moving  average  and  single  exponential  smoothing 
were  fit.  The  purpose  of  this  was  to  contrast  these  results  with  the  regres¬ 
sion  models  that  were  developed.  The  single  exponential  smoothing  is  simply 
based  on  averaging  past  values  of  a  time  series  in  an  exponential  manner. 

The  time  series  fit  was: 

SUMX  ,  ■  Of  SUMX  +  (1  -  a)  SUMX  (1) 

t+1  t  t 

or 

SUMXt+1  *  SUMXt  +  a  (SUMXt  -  SUMX^  (2) 

The  smoothing  is  achieved  by  giving  exponentially  decreasing  weights  to  past 
observations.  Equation  (2)  shows  that  the  forecast  for  period  t  +  1  is  equal 
to  the  forecast  for  the  previous  period  plus  an  adjustment  which  is  equal  to 
the  error  in  the  previous  forecast  times  the  weighting  factor  a.  The  value 
of  a  which  yielded  the  model  with  the  smallest  mean  square  error  (MSE)  was 
chosen. 

For  the  moving  average  models,  the  number  of  terms  yielding  the  smallest 
MSE  were  included  in  the  moving  average.  The  model  is  simply: 


where  k  is  the  number  of  terms  in  the  moving  average. 

All  of  the  time  series  results  were  obtained  using  the  SYBIL/RUNNER  time 
3 

series  package  .  The  results  are  shown  in  Figures  1-4.  The  mean  square 
error  (MSE)  as  well  as  the  mean  absolute  percentage  error  (MAPE)  are  shown 
on  the  plots.  These  plots  show  both  the  observed  and  predicted  results  using 


Moving  Average  Model  for  SUMX  -  Moody  AFB 


SUPPLY 


Exponential  Smoothing  Model  for  SUMX  -  Moody  AFB 


SUPPLY  PERFORMANCE  INDICATORS  BERGSTROM  AFB 


Moving  Average  Model  for  SUMX  -  Bergstrom  AFB 


all  27  observations.  * 


Linear  regression  models  for  SUMX  were  also  fit.  The  results  were  poor 
in  terms  of  the  amount  of  variation  in  the  data  explained,  however,  still 
clearly  superior  to  the  time  series  models  as  can  be  seen  by  comparison  with 
Figures  5  and  6.  The  respective  models  are  as  follows: 

Moody  AFB :  SUMX  20.08  +  .1826  REPA  -  .1063  REPGA 
Bergstrom  AFB:  SUMX  =  49.93  +  .1417  REPB  -  .5150  REPGA 
It  is  interesting  to  note  the  similarities  in  these  models.  Both  use  a  "REP" 
due  out  measure  as  well  as  REPGA.  The  linear  correlation  bewteen  RF.PA  and 
REPB  for  the  Moody  AFB  data  is  0.5  which  is  significant  at  a  c  .004. 

The  statistical  summary  of  these  regressions  is  given  below: 

2 

Model  Mean  Square  Residuals  R  F 

Moody  8.57  0.30  5.11 

Bergstrom  20.64  0.26  4.13 

The  mean  square  due  to  residuals  compares  favorably  with  the  equivalent  MSE 
from  the  time  series  models.  The  standard  error  is  2.93  for  the  Moody  model 
and  4.54  for  the  Bergstrom  model.  There  was  no  evidence  in  the  residual 
analysis  to  suggest  transformations  of  variables.  Nonlinear  models  of  the 
form: 


A  bl  b2 

SUMX  =  bQ  EOQGA  EOQA 


REPTII 
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wecealso  fit  for  all  observations  as  well  as  each  fiscal  year.  These  models 

were  discarded  since  they  generally  offered  no  improvement  over  the  simple 

linear  models  and  are  considerably  more  difficult  to  interpret.  These  results 

are  included  in  the  addendum  to  this  report. 

Considerably  better  fits  were  obtained  when  models  were  fit  to  a  specific 

fiscal  year,  either  FY79  or  FY80.  The  specific  FY  models  were  quite  different 

2 

from  each  other,  however,  R  val  ues  as  high  as  0.98  could  be  obtained  in  this 

*Note:  Time  series  models  for  NMCS  were  similar  to  these  and  are  included 

in  the  Appendix. 
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Actual  and  Predicted  Values  of  SUMX  -  Moody  AFB 


SUPPLY 
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manner. 


The  FY  79  and  FY  80  models  are  shown  along  with  the  actual  value  of 


SUMX  for  both  the  Moody  and  Bergstrom  data  in  Figures  7  and  8.  The  models  are 
given  below: 

Moody  AFB: 

FY  79:  SUMX  =  11.8  +  .92  EOQB  +  .368  EOQC  +  .060  EOQD  +  .113  REPC  -.078*REPD 


FY  80:  SUMX  =  122.6  -  .886  EOQNA  -r  1.23  EOQC  -  .236  RE PGA  +  .267  REPA 
-.253*  REPB  -  .0012  REPTII 

Bergstrom  AFB : 

FY  79:  SUMX  =  -133  +  1.66*  EOQNA  +  .052  EOQA 

FY  80:  SUMX  =42.2  -  .077*  EOQA  -  .423  REPGA  +  541  REPA  +  .280  REPB 

-1.19*  REPC  +  .072  REPD 

One  of  the  problems  with  these  models  is  that  one  or  more  of  the  coeffi¬ 
cients  has  a  sign  which  appears  counter-intuitive  (coefficients  with  asterisks). 

For  example,  the  Bergstrom  FY  79  model  indicates  that  an  increase  in  the  net 
availability  of  EOQ  type  items  would  result  in  an  increase  in  the  number  of 
aircraft  with  degraded  capability.  However,  these  models  are  good  fits  to  the 
actual  value  of  SUMX  in  a  given  FY.  The  summary  statistics  for  these  regres¬ 
sion  models  demonstrate  this  point. 


Model 

Mean  Square 
(Residuals) 

R2 

*1 

F 

Moody  FY  79 

0. 1214 

.98 

.97 

68.  24 

Moody  FY  80 

2.6736 

.  90 

.81 

10.32 

Bergstrom  FY 

79 

18.4506 

.45 

.32 

3.62 

Bergstrom  FY 

80 

2.6222 

.96 

.90 

17.73 

These  models  were  selected  using  a  forward  selection  technique  with  the  cri¬ 
tical  partial  F  value  set  to  2.3,  which  is  roughly  equivalent  to  a  level  of 
significance  a  *  0.20.  Variables  were  added  to  the  model  as  long  as  the 
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Moody  AFR 
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n»j 


2  4  2 

partial  F  exceeded  2.2  and  as  long  as  the  adjusted  R  value  ,  i.e.,  R^  continued 

2  2 

to  increase.  R,  is  a  modification  of  R  that  accounts  for  the  number  of  vari- 
A 

2 

ables  in  the  model.  The  coefficient  of  multiple  determination  R  is  defined  as: 

2  SSE 

R  - 1  -sq 

where  SS^  is  the  sum  of  squares  due  to  residuals  and  SS^  is  the  total  connected 
sum  of  squares. 

> 

In  contrast  RT  takes  into  account  the  number  of  variables  in  the  model  since 
A 


RA  =  1 


SS£/(n-p) 

SST/(n-lj 


where  p  is  the  number  of  terms  in  the  model  and  n  is  the  number  of  observations. 

2  2 

Unlike  R  ,  R  mav  decrease  as  variables  are  added  to  the  model. 

A 

The  maximum  number  of  variables  permitted  in  the  model  was  6  since  only 
12  observations  were  used.  This  was  purely  judgmental;  however,  we  felt  that 
this  was  a  reasonable  limit. 

As  mentioned  above,  the  predictive  regression  models  change  drastically 
from  fiscal  year  to  fiscal  year.  To  determine  if  the  change  was  gradual  over 
time,  or  if  the  changes  were  just  as  drastic  over  small  time  intervals,  addi¬ 
tional  regression  models  were  developed  for  a  moving  twelve  month  window  for 
the  variables  of  interest,  SUMX  and  NMCS.  Twelve  months  were  chosen  to  mini¬ 
mize  the  impact,  if  any,  of  seasonal  variations,  funding  cycles,  or  operational 
requirements  exercises.  The  data  in  Tables  2  and  3  shew  the  wide  variation  in 
successive  models  when  10  of  the  12  data  points  are  identical.  This  lack  of 
stability,  together  with  the  multitude  of  contradictory  algebraic  signs,  leads 
to  concern  about  domination  of  these  dependent  variables  by  factors  outside 
the  model.  These  factors  may  be  as  simple  as  different  interpretations  to 


the  indicator  definitions  or  as  complex  as  major  policy /command  changes. 


TABLE  2.  Bergstrom  AFB  Regression  Models  ( a  =  .10) 


Data 

Period 

Variables  in  SUMX 

Model 

R^ 

Oct 

78 

- 

Sep 

79 

EOQA 

.28 

Dec 

78 

- 

Nov 

79 

E0QNA* **,  REPGA 

.50 

Feb 

79 

- 

Jan 

80 

REPTII 

.33 

Apr 

79 

- 

Mar 

80 

EOQTII,  REPC** 

.54 

Jun 

79 

- 

May 

80 

REPC**,  REPTII 

.62 

Aug 

79 

- 

Jul 

80 

REPTII 

.39 

Oct 

79 

- 

Sep 

80 

REPGA,  REPB 

.64 

Dec 

79 

- 

Nov 

80 

* 

_ 

Data 

Period 

Variables  in  NMCS 

Model 

r! 

Oct 

78 

- 

Sep 

79 

E0QNA**,  REPGA 

.60 

Dec 

78 

- 

Nov 

79 

EOQNA**,  REPGA 

.60 

Feb 

79 

- 

Jan 

80 

* 

— 

Apr 

79 

- 

Mar 

80 

* 

— 

Jun 

79 

- 

May 

80 

REPNA** ,  REPC**,  REPTII 

.77 

Aug 

79 

- 

Jul 

80 

EOQTII** 

.32 

Oct 

79 

- 

Sep 

80 

EOQGA**,  REPD** 

.57 

Dec 

79 

- 

Nov 

80 

EOQGA** ,  EOQTII** 

.75 

*  Model  could  not  be  formulated  at  the  o'  ■  .10  significance  level. 

**  Model  has  this  variable  with  an  intuitively  contradictory  algebraic  sign. 


TABLE  3.  Moody  AFB  Regression  Models  ( a  =  .10) 


Data 

Period 

Variables  in  SUMX  Model 

b! 

Oct  78 

-  Sep 

79 

EOQB,  EOQC,  E0QD,  REPC ,  REPD** 

.98 

Dec  78 

-  Nov 

79 

EOQNA,  E0QTII** 

.52 

Feb  79 

-  Jan 

80 

EOQGA** ,  EOQNA,  REPA,  REPTII** 

.83 

Apr  79 

-  Mar 

80 

* 

Jun  79 

-  May 

80 

* 

Aug  79 

-  Jul 

80 

REPGA ,  REPTII 

.56 

Oct  79 

-  Sep 

80 

REPA,  REPB** ,  REPTII 

.76 

Dec  79 

-  Nov 

80 

REPGA 

.44 

Data 

Period 

Variables  in  NMCS  Model 

r! 

Oct  78 

-  Sep 

79 

REPA 

.45 

Dec  78 

-  Nov 

79 

* 

Feb  79 

-  Jan 

80 

REPA 

.32 

Apr  79 

-  Mar 

80 

REPA 

.32 

Jun  79 

-  May 

80 

* 

Aug  79 

-  Jul 

80 

REPA,  REPTII 

.67 

Oct  79 

-  Sep 

80 

REPGA**,  REPA,  REPB**,  REPTII 

.88 

Dec  79 

-  Nov 

80 

* 

Model  could  not  be  formulated  at  the  o'  ■  ,10  significance  level. 


**  Model  has  this  variable  with  an  intuitively  contradictory  algebraic  sign. 
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These  results  serve  only  to  demonstrate  the  difficulty  of  developing  a 
robust  predictive  model.  As  a  general  rule,  we  observe  that  the  due-out 
variables  and  the  availability  variables  are  useful  indicators  to  predict 
operational  capability  ^evtls.  However,  the  dramatic  change  in  the  models 
makes  forecasting  questionable.  In  fact,  the  single  exponential  time  series 
models  do  a  better  job  of  forecasting  and  these  are  recommended  if  forecasting 
is  required. 

An  alternative  to  the  time  series  models  for  forecasting  would  be  to  develop 
a  weighted  regression  model  (using  perhaps  exponential  weighting)  and  then  to 
forecast  only  one  month  in  advance.  2Lt  (then  C1C)  Michelle  Johnson  worked  on 
this  only  briefly  during  a  Math  499  course  and  her  entire  report  is  cited  as 
reference  [S'1.  Figure  9,  which  will  serve  to  illustrate  this  technique  and 
compare  it  with  the  time  series  forecast,  is  included  here.  We  observe  that 
although  the  weighted  regression  model  forecast  follows  the  actual  observa¬ 
tions  more  dramatically,  it  also  misses  rather  dramatically  on  occasion  such 
as  for  July  and  October  1980. 

3 .  Search  for  the  Best  Causative  Model 

The  second  objective  of  this  research,  to  determine  supply  performance 
goals  that  will  achieve  a  stated  level  of  operational  capability, implies  that 
we  wish  to  determine  cuuse-effect  relationships  between  a  performance  indica¬ 
tor  and  a  set  of  explanatory  supply  measures.  Section  2  of  the  report  clearly 
does  not  address  this  issue.  Prediction  and  forecasting  are  discussed  primarily 
because  they  represent  work  that  was  accomplished  in  pursuit  of  this  second 
goal. 

A  causal  relationship  cannot  be  validated  without  additional  experimental 
evidence.  In  this  particular  case,  it  is  unlikely  that  one  would  ever  be  able 
to  control  the  system  in  such  a  way  as  to  obtain  this  information.  Thus,  when 
we  state  that  we  have  a  "causative  model"  we  simply  mean  that  we  have  a  model 
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\  -  OBSERVED  DATA 

*  □  EXPONENTIAL  FORECAST 

1 '  A  REGRESSION  FORECAST , 


Fig  9. 


Forecasts  for  6/80  -  12/80  Using  Time  .Series  and  Weighted 
Regression  Models  -  Bergstrom  AFB 


which  has  good  face  validity.  That  is,  the  relationships  among  the  variables 
seem  reasonable  and  have  good  intuitive  appeal.  There  is  no  attempt  to  vali¬ 
date  such  a  model  using  the  observations  provided  since  standard  statistical 
techniques  can  address  only  significant  correlations  rather  than  cause-effect 
relationships . 

Since  the  fiscal  year  models  provided  the  best  results  in  the  previous 
section,  we  decided  to  look  at  the  causative  models  by  fiscal  year  as  well. 

The  procedure  for  developing  these  models  was  somewhat  similar  to  the  sequen¬ 
tial  selection  procedure  used  in  Section  2,  except  that  the  sign  of  the  regres¬ 
sion  coefficient  was  examined  for  face  validity.  We  postulated  that  measures 
of  supply  performance  that  should  correlate  negatively  with  degraded  capability 
rates  were  the  measures  of  availability  of  parts  and  the  inventory  investment 
(EOQGA,  EOQNA ,  EOQTII,  REPGA,  REPNA,  REPTII).  The  remaining  measures,  the  num¬ 
ber  of  monthly  due-outs,  were  expected  to  have  a  positive  correlation  with  the 
degraded  capability.  Thus,  as  availability  or  inventory  investment  increases, 
we  would  expect  NMCS  or  SUMX  to  decrease,  and  as  the  number  of  due-outs  increase, 
we  expect  NMCS  or  SUMX  to  increase  also. 

The  following  causative  models  were  developed: 

Moody  AFB: 

A 

FY  79:  SUMX  =  56.28  +  .0983  REPA  -  .4984  EOQNA 

NMCS  =  24.65  +  .1392  REPA  -  .2648  EOQNA 

FY  80:  SUMX  =  32.36  +  .2606  REPA  -  .001361  REPTII 

NMCS  =  22.69  .1581  REPA  -  .001294  REPTII 

Bergs troin  AFB: 

A 

FY  79:  SUMX  -  2.387  +.0418  EOQA  +  .  2567  REPB 

A 

NMCS  -  37.95  -  .5308  REPGA  +  .1685  REPB 

A 

FY  80:  SUMX  -  78.93  -  .9465  REPGA  +  .1715  REPB 

A 

NMCS  =  6.225  +  .0746  REPB 


22 


These  models  are  somewhat  more  satisfying  than  the  previous  models  because  of 
their  consistency.  The  set  of  explanatory  variables  for  Moody  AFB  always  con¬ 
tain  REPA  and  utilize  EOQNA  for  FY  79  and  REPTII  for  FY  80.  The  models  for 
Bergstrom  AFB  always  contain  REPB  and  two  of  them  use  REPGA.  With  the  excep¬ 
tion  of  the  Bergstrom  FY  80  model  for  NMCS  and  the  FY  79  model  for  SUMX,  all 
models  contain  one  measure  of  due-outs  and  one  measure  of  availabill ty /invest¬ 
ment.  The  summary  statistics  for  these  models  show,  however,  that  they  do  not 
fit  the  data  as  well  as  the  predictive  models. 

"CAUSATIVE”  MODELS  SUMMARY 


Mean  Square 

9 

r2 

Model 

(Residuals) 

R2 

_A 

F 

Moody  FY  79  (SUMX) 

2.51 

.46 

.34 

3.89 

Moody  FY  79  (NMCS) 

1.63 

.54 

.44 

5.25 

Moody  FY  80  (SUMX) 

6.47 

.62 

.54 

7.39 

Moody  FY  80  (NMCS) 

5.07 

.56 

.46 

5.69 

Bergstrom  FY  79  (SUMX) 

19.79 

.41 

.27 

3.07 

Bergstrom  FY  79  (NMCS) 

5.22 

.58 

.49 

6.33 

Bergstrom  FY  80  (SUMX) 

11.64 

.64 

.56 

8.05 

Bergstrom  FY  80  (NMCS) 

5.74 

.25 

.18 

3.35 

The  above  results  show 

that  the  NMCS  model  is 

superior 

to  the 

SUMX  model 

(in  terms  of  its  ability  to 

i  explain  the  variation 

in  the  data)  for 

FY  79. 

However,  the  SUMX  model  is 

better  for  FY  80.  We 

hesitate 

to  strike  too  many 

2 

conclusions  from  these  data  since  we  are  dealing  with  relatively  low  R  values. 
The  residual  analysis  for  these  models  showed  no  evidence  that  transformations 
of  the  independent  variables  were  called  for.  A  comparison  of  the  predicted 
regression  values  with  the  actual  data  is  shown  in  Figures  10-13. 

An  examination  of  these  plots  indicates  that  the  overall  fits  are  somewhat 
better  for  Moody  AFB.  The  primary  difficulty  with  the  Bergstrom  models  appears 
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Fig  13.  "Causative"  Regression  Models  for  NMCS  by  FY  -  Bergstrom  AFB 


to  occur  during  the  last  half  of  calendar  year  1980.  It  is  interesting  to 
note  also  that  the  data  for  Moody  AI'B  during  FY  79  and  FY  80  exhibit  no  par¬ 
ticular  trend;  while  for  Bergstrom  AFB  there  is  a  general  downward  trend  in 
the  dependent  variables  for  FY  79,  but  a  contrasting  upward  trend  for  FY  80. 
This  is  more  pronounced  for  NMCS  (see  Figure  13). 

The  "causative"  models  for  the  complete  set  of  27  observations  were  the 
same  as  the  "predictive"  models  previously  mentioned.  For  convenience,  we 
repeat  them  here: 

Moody  AFB : 

SUMX  =  20.08  +  .1826  REPA  -  .1063  RE PGA 

Bergstrom  AFB : 

A 

SUMX  -  49.93  +  .1417  REPB  -  .5150  REPGA 
Although  a  model  for  NMCS  was  obtained  for  Moody  it  was  inferior  to  the  SUMX 
model,  and  no  significant  regression  was  obtained  for  NMCS  using  the  Bergstrom 
data.  The  R"  values  for  the  above  overall  models  were  0.30  and  0.26  respec¬ 
tively,  considerably  lower  than  the  specific  fiscal  year  models. 

These  models  are  not  very  impressive  in  terms  of  their  ability  to  explain 
the  variation  in  operational  capability  levels.  However,  they  do  demonstrate 
that  there  is  a  consistent  relationship  between  due-outs  and  availability 
measures  and  the  operational  capability  measures  of  NMCS  and  SUMX  as  we  would 
intuitively  expect.  As  previously  mentioned,  there  is  no  way  to  validate  a 
cause-effect  relationship,  although  there  would  seem  to  be  one.  Unfortunately, 
the  cause-effect  relationships  are  no  doubt  complex.  One  method  of  analyzing 
these  relationships  is  known  as  causal  path  analysis^.  [See  also  reference  2, 
pages  383-397.] 

4.  Causal  Path  Analysis 

To  attempt  to  find  causal  relationships  between  the  supply  performance 
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indicators  and  the  measures  of  performance,  we  exercised  the  statistical 
package  for  the  social  sciences  capability  in  a  proposed  model  of  causal 
relationships.  Causal  path  analysis  requires  two  major  assumptions: 

1)  the  causal  relationships  (perhaps  weak)  are  known. 

2)  the  variables  in  the  analysis  are  causally  closed. 

Satisfying  the  first  relationship,  given  the  apparent  random  nature  much  of 
this  data  exhibits,  is  not  entirely  possible.  Correlations  exist  among  many 
of  the  performance  indicator  variables  which  serve  to  disguise  causal  rela¬ 
tionships.  The  economic  order  quantity  (EOQ)  and  reparable  (REP)  variables 
naturally  divide  the  independent  variables  into  two  classes.  Total  inventory 
investment  (EOQ  or  REP)  snems  to  be  an  externally  influenced  variable  that 
would  have  a  direct  effect  on  availability  and  due-outs.  Availability  and 
aue-outs  ought  to  affect  a  performance  measure  like  SUMX.  This  type  of  analy¬ 
sis  led  to  the  postulation  of  the  model  presented  in  Figure  14,  with  the  causal 

order  specified  by  the  arrows  between  boxes.  The  stand  alone  arrows  indicate 

the  sum  of  all  other  causes  for  the  variable  in  the  box.  These  sources  are 
essentially  random  error  if  the  chosen  model  is  adequate.  If  the  model  is 
inaccurate,  these  sources  are  the  dominant  causes  and  the  model  is  not  causally 
closed.  The  P  represent  the  standardized  regression  coefficients,  which  are 
used  for  ease  of  interpretation.  In  Figure  14,  a  unit  change  in  Y^  should 
bring  about  a  change  m  Y2.  A  unit  change  of  Y^  will  cause  a 

P31  +  ^P21^  ^P32^  chan8e  in  Y3>  and  30  on- 

Figures  15  through  20  display  causal  path  anfclysis  results  for  the  basic 

model,  which  was  somewhat  arbitrarily  chosen  midway  through  this  study  based 
on  the  variable  relationships  understood  at  that  time.  Gross  availability  was 
chosen  over  net  availability.  Due-out  reason  B,  out-of-stock  because  of  policy 
or  management  decision,  was  chosen  over  the  others.  Many  of  the  results  from 
the  regression  studies  are  apparent  in  these  figures.  These  include 
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Causal  Path  Basic  Model 
Bergstrom  AFB,  FY  80 


Bergstrom  AFB 


Figure  17.  Causal  Path  Basic  Model  - 
Bergstrom  AFB,  FY  79  &  FY 


Moody  AFB 

FY  79  *946 


Figure  18.  Causal  Path  Basic  Model 
Moody  AFB,  FY  79 


Causal  Path  Basic  Model 
Moody  AFB ,  FY  80 


1)  The  error  quantities  are  too  large,  indicating  that  there  are  perhaps 
other  quantities  causing  the  trends  we're  seeing. 

2)  The  data  is  not  stable  from  one  period  to  another.  There  isn't  con¬ 
sistency  from  FY  79  to  FY  80.  Combining  the  24  data  points  doesn't  provide 
any  uniform  improvement. 

3)  Standardized  regression  coefficients  leave  intuitively  intolerable 
algebraic  signs.  In  Figure  15,  why  should  an  increase  in  EOQGA  (desirable) 
"cause"  an  increase  (.141)  in  SUMX  (a  measure  of  degraded  capability  and  un¬ 
desirable)  ? 

To  try  to  improve  on  the  basic  model,  we  examined  the  correlation  matrices 
for  the  two  bases  and  two  fiscal  years  to  look  fcr  better  variable  choices,  but 
still  within  the  framework,  of  the  basic  model.  For  EOQ  and  REP,  either  gross 
availability  or  net  availability  was  chosen  based  upon  the  correlations  within 
the  model,  i.e.,  with  TII  and  SUMX.  A  similar  estimate  of  the  best  variable  was 
chosen  for  the  due-outs  (reasons  A,  B,  C,  and  D).  The  results  for  these  models 
are  shown  in  Figures  21  through  24. 

The  additional  effort  in  attempting  to  find  stronger  causal  relationships 
did  not  produce  a  uniform  improvement.  Most  of  the  criticism  of  the  basic 
models  can  be  applied  to  these  selected  models. 

The  entire  effort  at  causal  path  analysis  should  be  viewed  as  one  pro¬ 
viding  additional  insight  into  the  data,  but  not  as  one  that  found  strong, 
causal  relationships.  The  concerns  and  conclusions  from  the  predictive  re¬ 
gression  models  also  apply  in  the  search  for  causal  path  models. 

Although  we  could  have  continued  to  search  for  causal  path  models  that 
would  exhibit  more  desirable  path  coefficients,  we  felt  that  it  would  not  add 
substantially  to  our  understanding  to  continue.  An  example  of  how  we  might 
interpret  one  of  the  models  is  as  follows  for  Figure  23  -  the  FY  79  Moody  AFB 
model.  We  might  argue  that  the  paths  between  EOQTII  and  EOQD  and  REPTII  and 


Causal  Path  Selected  Model 
Bergstrom  AFB,  FY  79 


Bergstrom  AFB’ 

FY  80  .948  .939 


Causal  Path  Selected  Model 


-.046  .  c  .064 

EOQTII  EOQNA  REPNA  REPTII 


Figure  24.  Causal  Path  Selected  Model 
Moody  AFB ,  FY  80 


REPA  should  be  eliminated  as  there  may  be  no  direct  relationship  there.  This 
would  take  care  of  the  apparent  incorrect  path  sign.  The  fact  that  there  are 
incorrect  signs  between  REPTII  and  REPNA  and  REPNA  and  SUMX  may  be  due  to  our 
inability  to  statistically  determine  a  correct  model  due  to  large  error  sources 
in  REPNA.  The  other  signs  appear  correct.  Finally,  we  note  that  over  65%  of 
the  variation  in  SUMX  still  must  be  attributed  to  other  sources. 

5.  Summary  and  Conclusions 

The  original  objective,  to  find  statistically  significant  correlations 
between  operational  capability  levels  and  supply  performance  measures,  was 
addressed  by  developing  predictive  and  causative  regression  models.  Although 
not  specifically  requested,  we  also  discussed  the  use  of  time  series  models 
for  forecasting  and  compared  these  with  the  regression  models.  Since  the 
secondary  objective  involved  setting  supply  performance  level  goals  in  order 
to  achieve  stated  levels  of  operational  performance,  we  looked  for  cause-effect 
relationships  using  causal  path  analysis.  There  was  no  statistical  way  to 
verify  any  of  these  cause-effect  relationships,  but  we  did  apply  intuition 
and  common  sense  in  order  to  achieve  models  that  had  good  face  validity. 

Due-out  measures  as  wall  as  availability  measures  clearly  influence  the 
operational  capability  levels.  Unfortunately,  the  interrelationships  among 
these  various  measures  and  the  large  source  of  error  (i.e.  unmeasurable  fac¬ 
tors  that  influence  the  operational  capability  levels)  made  it  impossible  to 
develop  a  robust  model,  even  for  a  particular  AFB. 

Two  measures  of  operational  capability  were  discussed  in  this  report: 

NMCS,  the  non-mission  capable  rate  due  to  only  supply  reasons  and  SUMX 
(NMCS  +  PMCS  +  NMCB  +  PMCB),  the  mission-capability  degraded  rate  due  to 
supply  reasons.  We  were  able  to  generally  develop  better  models  for  SUMX. 

This  may  perhaps  be  attributable  to  the  fact  that  SUMX  as  the  sum  of  the  four 
given  operational  indicators  may  represent  an  averaging  of  the  different 


sources  and  cancel  out  any  effects  due  to  improper  or  inconsistent  identifi¬ 
cation. 

Although  a  general-purpose  model  could  not  be  developed,  the  following 

results  demonstrated  some  consistency  among  the  models.  For  the  fiscal  year 

causative  models,  REPA  always  appeared  as  the  due-out  variable  of  interest  in 

the  Moody  models.  Similarly,  REPB  always  appeared  as  the  primary  due-out 

variable  in  the  Bergstrom  models.  Furthermore,  six  out  of  eight  of  these 

models  included  an  availability  or  investment  measure  as  well  as  the  due-out 

2 

measure.  No  model  had  more  than  two  supply  measure  variables.  The  R  values 
for  these  models  ranged  from  .25  to  .64. 

An  overall  model  for  SUMX  for  each  base  was  developed.  These  models  both 
used  REPGA  as  the  availability  measure  and  either  REPA  or  REPB  as  the  due-out 
measure.  We  were  surprised  at  the  overall  dominance  of  the  reparable  variables 
in  these  models.  However,  we  hesitate  to  draw  any  specific  conclusion  from 
this  without  a  deeper  understanding  of  these  measures.  The  term  causative 
was  used  only  to  refer  to  the  fact  that  the  model  had  good  face  validity  and 
to  contrast  it  from  che  purely  predictive  models  that  were  also  addressed.  In 
general,  a  causative  model  tended  to  have  one  due-out  measure  and  one  avail¬ 
ability  measure  in  it. 

The  predictive  models  fit  the  data  much  better,  but  offered  little  or  no 

interpretive  value  and  all  had  poor  face  validity.  They  were  generally  char- 

2 

acterized  by  more  variables  (usually  5)  and  had  much  higher  R  values  ranging 
from  .45  to  .98. 

A  causal  path  analysis  was  discussed  only  for  the  purpose  of  helping  to 
understand  the  possible  cause-effect  relationships  which  may  exist  among  the 
variables.  No  validation  was  offered  for  this  and  it  should  only  be  viewed  as 
a  descriptive  tool  that  may  lend  some  insight  into  such  relationships. 


We  were  unable  to  achieve  the  second  objective  of  this  work  which  was  to 
specify  specific  levels  of  supply  measures  which  should  be  met.  Such  a  re¬ 
sult  would  require  a  robust  cause/effect  model.  It  is  doubtful  that  this 
result  could  ever  be  achieved  with  the  data  that  was  provided  for  this  study. 

Finally,  two  areas  of  analysis  that  were  investigated  but  proved  unfruit¬ 
ful  include  lagging  and  advancing  (in  time)  several  of  the  independent  variables, 
and  averaging  the  monthly  data  by  fiscal  quarters  in  order  to  remove  seasonality 
effects.  These  techniques  provided  models  that  were  clearly  worse  than  the  ones 
discussed  in  this  report. 

6.  Recommendations  for  Further  Study 

We  feel  that  there  is  not  much  more  that  can  be  investigated  with  the  pre¬ 
sent  data  set  with  the  possible  exception  of  developing  weighted  regression 

models.  Some  type  of  weighting  (perhaps  exponential)  of  the  observations  may 

2 

provide  the  means  to  come  up  with  a  robust  model  with  improved  R  values  and 
would  be  an  attractive  alternative  to  the  specific  fiscal  year  models  developed. 

If  more  data  becomes  available  (at  least  two  more  fiscal  years)  a  further 
investigation  of  the  nature  of  the  changes  in  the  specific  fiscal  year  models 
may  provide  some  additional  insight  into  these  issues. 

Insight  into  managerial  policies  that  may  change  from  one  fiscal  year  to 
another  might  also  provide  an  additional  source  of  information  with  which  to 
explain  the  change  in  fiscal  year  models.  If  such  information  were  available 
further  work  on  regression  models  may  prove  useful. 


APPENDIX 


List  of  Raw  Data  (Moody  AFB) 

List  of  Raw  Data  (Bergstrom  AFB) 
Data  Coding  and  Format 
Time  Series  Models  for  NMCS 
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Data  Coding  and  Format 


Each  data 

Record  1: 

Record  2: 


observation  contains  two  records  with  the  following  information: 


Month,  year,  variable  type,  EOQGA,  EOQNA,  EOQA,  EOQB,  EOQC,  EOQD, 
EOQTII 

Base,  variable  type,  RE PGA ,  REPNA,  REPA,  REPB ,  REPC,  REPD,  REPTII, 
NMCS,  NMCB ,  PMCS ,  PMCB 


The  data  is  formatted  for  SPSS  as  follows: 


FIXED  (IX,  2F2.0, 
9X,  2F6.2, 


4X,  2F6.2,  2F4.0,  F3.0,  F4.0,  F8.1/ 
2F4.0,  F3.0,  F4.0,  F8.1,  4F6.2) 
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Time  .Series  Models  for  NMCS 

a.  Single  Exponential  Smoothing: 

*  A 

NMCS  =0.1  NMCS  +0.9  NMCS 
t+1  t  t 

Mean  Square  Error  (MSE):  7.9 

Mean  Absolute  Percentage  Error  (MAPE):  31.0% 

b.  The  best  moving  average  model  for  NMCS  was  a  5-term  moving  average  with 
MSE:  8.0 

MAPE:  32.9% 
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